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Background/context: 

Description of prior research , its intellectual context and its policy context. 



Over the past five decades, the federal government and most states have invested heavily in 
providing publicly-funded child care and early education opportunities for three- and four-year- 
old children from low-income families. Policy makers and parents want to identify the level or 
threshold in quality of teacher-child interaction related to better child outcomes to most 
efficiently use child care to improve school readiness. 

Most of the literature has examined linear associations, yielding findings that higher quality is 
better and lower quality is worse (Vandell, 2004), but identification of thresholds in the 
association between quality and child outcomes has been a goal of researchers and policy makers 
for several reasons. A primary goal has been to identify levels in the association between quality 
and child outcomes at which the linear association begins to asymptote or level off, above or 
below which there is little evidence of increases in learning associated with increased in quality. 
A threshold that indicated that the quality-outcome association level off above a given level of 
quality would suggest that policies should focus on improving quality up to that threshold level, 
but improving quality above that point may not be necessary for improving child outcomes. 
Policy to address this goal would invest in lower or average quality classrooms while leaving 
classroom with quality scores above the threshold alone. In contrast, it is possible a threshold 
could define the minimum level at which a positive association between quality and outcomes is 
observed. In this scenario, there may be no detected relation between quality and outcome gains 
until quality reached a certain point on the scale; in other words, learning did not take place until 
classrooms demonstrated a minimal level and after that minimum, gains in learning increased as 
quality increased. This form of threshold effect would suggest that it is especially important to 
ensure that children experience at least the minimum level of quality child care in order for those 
experiences to be related to improved child outcomes. It would point perhaps to not allowing 
vouchers to pay for care that was below the threshold, while also incentivizing teachers above 
the threshold to continue to improve. One can see how the examination of thresholds may have 
considerable implications for the efficient and effective expenditure of funds, not only in relation 
to providing access to quality of a certain minimal level, but also in the targeting of funds to 
improve quality. 

Whether there was any evidence of thresholds in the association between quality of center-based 
child care and gains in academic and social skills was examined in 5 large studies of child care 
experiences of children from low-income families. Using a quadratic or spline regression 
approaches, findings suggested that teacher sensitivity was related to higher levels of social 
competence and lower levels of behavior problems only in high-quality classrooms 

Setting: 

Description of where the research took place. 



All data included in these secondary data analyses were collected in preschool center-based care. 
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Population / Participants / Subjects: 

Description of participants in the study: who (or what ) how many, key features (or characteristics). 



We examined the association between child care quality and child outcomes among low-income 
children. We selected five data sets that included child care quality and child outcome 
assessments for at least 100 children observed in at least 50 classrooms. The data included in 
this analysis from the NICHD Study of Early Child Care and Youth Development (NICHD 
SECCYD; NICHD ECCNR, 2003) and Cost, Quality, and Outcomes Study (CQO; Peisner- 
Feinberg & Burchinal, 1997) examined 4-5 year-old children in center-based community 
settings. The National Center for Early Development and Learning (NCEDL) examined 
children attending randomly selected state pre-kindergarten programs in 1 1 state chosen because 
they had mature large pre-kindergarten programs (Howes et al., 2008). The Head Start 
Experience Survey (FACES) from 1997 and 2000 provided a representative sample of children 
attending Head Start during those years.. The complete data were available for the first three 
studies and the public release data sets were acquired for the FACES data sets. 

The NICHD SECCYD was designed to examine the relationship between child care experiences 
and characteristics and children’s developmental outcomes. The participating children were a 
conditional random sample selected shortly after birth during hospital visits at ten locations 
across the U.S. The locations were associated with the 10 principle investigators, but include 
sites in the Northeast, Southeast, Midwest, and West. Data were collected in whatever care and 
education setting the children attended. The current analyses include ECE setting and child 
assessments 54 months. For more information on this study’s methodology, see 
https ://secc. rti.org or a report by NICHD study (NICHD ECCRN, 2006). 

The CQO was conducted in child care centers in four states. The centers were randomly selected 
from lists of for-profit and not-for-profit centers in the greater Los Angeles area of California, 
the Frontal Range area of Colorado, the greater Hartford area of Connecticut, and greater 
Greensboro area of North Carolina. Costs and quality of care in a randomly selected 
infant/toddler and preschool classroom were carefully assessed in 101 centers. Up to 10 children 
in the preschool classrooms were randomly selected and their academic, language, and social 
skills were assessed in the spring of their second-to-last year of child care (see Peissner-Feinburg 
& Burchinal, 1997 for details). 

NCEDL conducted two studies of state-funded pre-kindergarten: the Multi-State Study of Pre- 
Kindergarten and the Study of State-Wide Early Education Programs (SWEEP). The goals and 
methodologies of the two studies were largely similar, so they have been combined for the 
current analyses. Both studies sought to describe state-funded pre-kindergarten programs in 
states that had large, well-established programs. In all, 11 states participated. In each state, sites 
were randomly selected from lists provided by the states of sites providing state funded pre- 
kindergarten. Then, in each site, one classroom serving primarily 4-year olds was selected at 
random. Within each classroom, 4 children who were old enough to attend kindergarten the 
following year were selected. The current analyses include children’s assessments from the fall 
and spring of the pre-kindergarten year. For further details about the methodologies for these 
studies see Early et al. (2005). 

The primary purpose of the FACES was to describe the quality of Head Start programs in a 
nationally representative sample, toward the goal of implementing a system of program 
performance measures and improving accountability for Head Start programs. The sample 
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comprising this longitudinal dataset consisted of 63 randomly selected Head Start programs, 
stratified by census region, percent minority, and urbanicity. The FACES cohorts from 1997 and 
2000 feature four phases of data collection and follow 4-year old Head Start children from 
program entry through the spring of kindergarten. The current analyses are restricted to children 
who were in 4 year-old classrooms in spring of either 1997 or 2000 and include children’s 
assessments from the fall and spring of their first year of Head Start. For more information on 
this study and its methodology, see Zill and Resnick (2005) and Zill et al. (2003). 

Measures 

Five measures ofECE quality were collected across the studies. The Early Childhood 
Environment Rating Scale (ECERS; Harms & Clifford, 1980) or the Early Childhood 
Environment Rating Scale-Revised (ECERS-R; Harms, Clifford, & Cryer, 1998) was collected in 
all studies but SECCYD. It is a widely used measure of globed classroom quedity, specificcdly 
designed for use in classrooms serving children between 2% and 5 years of age. Scores on the 
ECERS-R range from 1-7 with 1 indicating “inadequate” quality, 3 indicating “minimal” 
cjuality, 5 indicating “good” cjuality, and 7 indicating “excellent” cjuality. The Observational 
Record of the Caregiving Environment (ORCE) was designed specifically for the SECCYD to 
assess the quedity of caregiver-chilel interaction experienced by indivielued children. 

Observations of child care ejuality used in the present analyses were made during a single half- 
day visit when the child was 54 months of age. The quality measure used for the current project 
is the positive caregiving rating composite, the mean of 4-point ratings of caregivers’ 
sensitivity/responsivity, stimulation of cognitive development, intrusiveness ( reflected ), and 
detachment (reflected). Scores range from 1 ( unresponsive or harsh care giving) to 4 ( frequent 
responsive and stimulating caregiving). The Caregiver Involvement Scale (CIS; Arnett, 1989) 
was collected in CQO and the FACES studies. It is an observational scale consisting of 26 items 
reflecting teacher sensitivity, harshness, and detachment that are rated on a 1-4 scale indicating 
how characteristic they are of the teacher, from not at cdl (1) to very much (4). Psychometric 
analyses suggest a single factor most parsimoniously represented these data ( Cronbach ’s a = 
.93). The Classroom Assessment Scoring System (CLASS; Pianta, La Paro, & Hamre, 2008) 
was used in NCEDL to rate teacher-child interactions in terms of emotional support, in terms of 
sensitivity and responsiveness p, and instructional support in terms of developing concepts and 
providing productive feedback. 

The projects also used a variety of measures to assess child outcomes (see Table 3). All of the 
projects but CQO administered the same battery of assessments in the fall and spring. All studies 
measured language. CQO, NCEDL, and the FACES studies used the Peabody Picture 
Vocabulary Test-Revised (PPVT; Dunn & Dunn, 1997. The SECCYD used the Preschool 
Language Scale-3 (PLS-3; Zimmerman, Steiner, & Pond, 1992). According to the developers, 
both measure receptive language reliably for 3-5 year-olds (PPVT 92 -.98; PLS-3 .89-. 92). All 
studies administered two subtests from the Woodcock- Johnson Tests of Achievement (WJ), the 
Letter-Word Identification subtest to measure early reading skills and the Applied Problems 
Subtest to measure early math skills. NCEDL used the Woodcock-Johnson III (Woodcock, 
McGrew, & Mather, 2001). According to the developer, reliability for 3- to 5-year-old children 
range .92 to .99 for Letter-Word and .91 to .94 for Applied Problems. All of the language and 
academic achievement measures are individually-administered standardized tests with mean of 
100 and standard deviation of 15 in the norming sample. 
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The five studies administered a variety of measures of social/emotional development. The 
SECCYD had teachers complete the Child Behavior Checklist (CBCL; Achenbach, 1991), a 
widely used measure of internalizing and externalizing behavior problems. The SECCYD 
measured social skills with the Social Skills Rating System (SSRS; Gresham & Elliott, 1990; test- 
retest reliability of.75 to .88). The CBCL has a mean of 50 and standard deviation of 10 in the 
norming sample. CQO teachers rated each child on the Classroom Behavior Inventory (CBI; 
Schaefer, Edgerton, & Aaronson, 1978). Factor analysis of the 10 CBI scale scores from the first 
year resulted in three factors (75% of the variance): yielding a task orientation factor (a = .84), a 
social skills factor (a = .65), and a problem behaviors factor (a = .77). CBI scales are computed 
as the mean of items with a maximum score of 5 and minimum score of 1. NCEDL teachers 
completed the Teacher-Child Rating Scale (TCRS; Hightower et al., 1986; NCEDL), a 
behavioral rating scale that assesses children’s social competence and problem behaviors. The 
Social Competence scale was computed as the mean of 20 items and had a Cronbach’s a of .95. 
The Problem Behavior scale was computed as the mean of 18 items and had a Cronbach’s a of 
.91. TCRS scales are also computed as the mean of items with a maximum score of 5 and 
minimum score of 1. The teachers in the two FACES studies completed the 28-item Behavior 
Problems Index (Zill, 1985) to assesses problem behaviors related to emotional status, school 
behavior, and interpersonal relationships with items drawn from several other child behavior 
scales (e.g., CBCL, SSRS). Zill reported 2-week test-retest reliability of .92. 



Research Design: 

Description of research design (e.g., qualitative case study, quasi-experimental design, secondary analysis, analytic 
essay, randomized field trial). 



We asked whether the reason that the associations between observed quality and child outcomes 
were so modest is because the association is nonlinear. We tested this hypothesis with regression 
analyses that included quality as both linear and quadratic terms (i.e., quality x quality) and site, 
maternal education, ethnicity, and gender as covariates. 



Findings / Results: 

Description of main findings with specific details. 

First, we examined the quadratic regression. Results are shown in Table 2, listing the 
coefficients and standard errors in parentheses for the models in which the quality squared term 
was significant. Quadratic associations obtained in the analyses of the FACES 1997 data. These 
regressions provide a hint that quality may be more strongly related to outcomes when quality is 
in the higher quality range. The positive quadratic associations indicate that the association 
between quality and outcomes is stronger at higher levels of quality whereas a negative quadratic 
association indicates the association is stronger at lower levels of quality. In one study, FACES 
1997, ECERS scores were positively related to language scores when quality was in the good to 
high range. In three studies, NCELD, SECCYD, and FACES 1997, quality was more strongly 
related to math skills when quality was in the good to high range. In one study, NCEDL, the two 
quality measures were more strongly related to reading skills when they were in the good to high 
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range. However, in another study, CQO, quality was more strongly related to reading when 
quality was in the low to average range. Finally, in two studies, NCEDL and FACES 1997, 
quality was more strongly negatively related to behavior problems when quality was in the good 
to high range. 

Second , a spline regression approach was used with NCEDL to test whether the linear 
regression of child outcomes onto child care quality for low- or average-quality programs was 
different than that for high- quality programs. A repeated-measures hierarchical linear model 
was fit to account for nesting of children within classrooms. Therefore, there was one residual 
term that has taken into account the clustering within classroom and an independent residual 
term that represented error in the individual children’s scores. The model estimated separate 
“splines” or linear regressions for programs considered to be lower quality and higher quality, 
thereby estimating one slope for describing the association between quality and outcomes for 
programs in the higher-quality range and another slope for programs in the lower-quality range. 

Emotional Support. The spline regressions tested whether Emotional Support predicted the six 
outcomes overall and whether the magnitude of prediction was stronger in higher-quality 
classrooms (i.e., classrooms with a score of 5 or higher) than in lower-quality classrooms. 
Results, indicated that Emotional Support was a more positive predictor of social competence 
and negative predictor of behavior problems in classes in the high range on Emotional Support 
than in classes in the low/medium range. Effect sizes were plotted in Figure 1. Emotional 
Support was more strongly positively related to teacher ratings of social competence in high- 
quality ( B = .09, d = .08) than in moderate-to-low-quality classrooms (. B = -.01, d = .01). 
Similarly, Emotional Support was more strongly negatively related to teacher ratings of behavior 
problems in high-quality ( B = -.10, d = -.12) than in moderate-to-low-quality classrooms ( B = 

.03, d = .04). Emotional Support predicted these two social outcomes in the hypothesized 
direction only in the higher-quality classrooms; it was a nonsignificant predictor of social 
competence and a positive predictor of behavior problems in moderate-to-low-quality 
classrooms. 

Instructional Quality. Instructional Quality was related to three of the four academic outcomes 
in these spline regressions. As shown in Figure 2, children who experienced higher-quality 
Instructional Quality (i.e., classrooms with a score of 3.25 or higher) tended to score higher on 
expressive language and math overall, but the magnitude of that association was stronger in 
higher-quality classrooms for reading, math, and expressive language. Instructional Quality 
predicted outcomes more strongly in higher-quality than lower-quality programs on expressive 
language (Higher: B = 3.97, d = .23; Lower B = 1.34, d = .08), reading (Higher: B = 5.54, d = 

.17; Lower B = .70, d = .02), and math (Higher: B = 8.30, d = .34; Lower B = 2.52, d = .02). 



Conclusions: 

Description of conclusions and recommendations based on findings and overall study. 



In conclusion, findings from this study suggest that moderately high- to high-quality classrooms 
may be necessary to optimally improve social skills, reduce behavior problems, and promote 
reading, math, and language skills. Unlike the expected threshold levels suggesting there was a 
“good-enough” level, these findings suggest that children may not obtain social and academic 
benefits from pre-kindergarten experiences unless the teacher maintains high-quality teacher- 
child interactions and at least moderate- to high-quality instruction. 
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Appendix B. Tables and Figures 

Not included in page count. 



Table 1 Descriptive information for the five studies on low-income children and their classrooms 







NICHD 


CQO 


NCEDL 


FACES 


FACES 






SECCYD 




Pre-K eval 


1997 


2000 






N=129 


N=140 


N=1465 


N=1493 


n=1739 


Background 
Maternal Education 


M (sd) 


12.6(1.8) 


13.0(1.7) 


11.8(1.9) 


11.7(1.9) 


11.9(1.8) 


Child Gender (male) 


% 


48% 


51% 


49% 


51% 


50% 


Child Age -months 
Child Ethnicity/Race 


M (sd) 


54 


51.1(4.4) 


60.6(2.3) 


55.4(6.3) 


54.0(6.5) 


African-American 




36.4% 


32.4% 


20.7% 


21% 


38% 


Hispanic 




9.3% 


13.7% 


36.0% 


36% 


29% 


White 




49.6% 


46.0% 


28.6% 


29% 


24% 


Other 

Quality Measures 




4.7% 


7.9% 


14.7% 


14% 


5% 


ECERS Total 


M (sd) 




4.2 (.9) 


3.8 (.8) 


5.1 (.6) 


4.9 (1.0) 


Language&Reason 


M (sd) 




4.38(1.40) 


4.5 (1.1) 


5.0 (1.0) 


5.0 (1.3) 


Interactions 


M (sd) 




4.62 (1.24) 


4.7 (1.5) 


5.0 (.7) 


5.6 (1.6) 


CIS Total 


M (sd) 




78.0(10.4) 




73.1 (11.1) 


72.8(13.0) 


CLASS Total 


M (sd) 






4.5 (.6) 






Emotional Support 


M (sd) 






5.5 (.7) 






Instructional Supp 


M (sd) 






2.1 (.8) 






ORCE Total 
Child Outcomes 


M (sd) 


2.9 (.6) 










PPVT 


M (sd) 




83.6(17.1) 


92.0(13.4) 


84.1 (15.1) 


84.6(16.7) 


PLS 


M (sd) 


86.2 (18.5) 










WJ-Letter-Word 


M (sd) 


91.9(12.8) 


94.7(10.9) 


99.5 (13.0) 


90.2 (10.1) 


92.5 (10.3) 


WJ-Applied Prob 


M (sd) 


93.4(15.7) 


96.6(12.8) 


96.0(12.2) 


84.9(17.7) 


86.6(17.5) 


CBCL Externalizing 


M (sd) 


53.3 (10.0) 










CBCL Internalizing 


M (sd) 


52.8(10.1) 










CPSC Socal Compet 


M (sd) 


98.4(16.6) 










CBI Task Orientation 


M (sd) 




3.3 (1.0) 








CBI Behavior Prob 


M (sd) 




2.7 (1.0) 








TCRS Social Compet 


M (sd) 






3.6 (.8) 






TCRS Behavior Prob 


M (sd) 






1.5 (.6) 






BPI Social Skills 


M (sd) 








16.4 (4.5) 


16.5 (4.6) 


BPI Behavior Prob 


M (sd) 








4.9 (4.5) 


5.5 (5.0) 
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Table 2. Regression Coefficients predicting child outcomes from quality as a linear and 
quadratic predictor 





ORCE 

SECCYD 


CQO 


ECERS/ECER-R 

NCEDL FACES 97 FACES00 


CLASS 

NCEDL 


Receptive Language 


NS 


NS 


NS 


NS 


NS 


Linear quality 








-18.62**(6.01) 




Quadratic quality 








1.92** (.60) 




Math-WJ Applied 




NS 


NS 


NS 


NS 


Prob. 


-52.65* 






-20.90*(8.00) 




Linear quality 


(23) 






2.1 1 **(.80) 




Quadratic quality 


9.32* (4.0) 










Reacling-WJ 


NS 




NS 


NS NS 


-14.0* (6.0) 


Letter/Word 




17.3* (6.6) 




1.70* (.70) 


Linear quality 




-1.8* (.8) 






Quadratic quality 












Socicd Skills 


NS 


— 


NS 


NS 1.76* 


NS 


Linear quality 








(.88) 




Quadratic quality 








NS 




Behavior Problems 


— 


NS 




NS 


NS 


Linear quality 






.35* 


-5.70** (2.00) 




Quadratic quality 






(.15) 


.64** (.20) 










.04* 












(.02) 






Note: all analyses ac 


ljusted for site, maternal education, gender, and ethnicity. * p <.05; ** 



< 0i ; *** pc. 001 
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Figure 1: NCEDL Spline regression results: Emotional Support effect sizes for high- and 
moderate-to-low-quality classrooms 
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Figure 2: NCEDL Spline regression results Instructional Quality effect sizes for moderate-to- 
high- and low-quality classrooms 
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